Towards Modeling Redundancy In Multimodal, Multi-party Tasks to Support Dynamic Learning

Our goal is to create computer systems that learn as easily as humans. As machines move closer to being observant and intelligent assistants for humans it is not enough that they rely on off-line models. They need to automatically acquire new knowledge as they are running, particularly by a single, natural demonstration. Current recognition systems need sophisticated models of both features and higher level sequential or combinatory patterns; for example, speech recognizers are trained at the feature level on large numbers of corpus-based examples of phonetic segments and then at higher levels are constrained by either rule-based symbolic or corpus-based statistical language models. However, whether recognition is rulebased or statistical, no system of static models can achieve full coverage. Natural language is replete with new words, new word patterns, and new topics. Thus, symbolic rulebased systems are notoriously brittle because they cannot handle new words or word patterns, while statistical models typically fail on test data that has little relation to the training corpus, as happens when dialogue shifts to a new topic area. So automatically acquiring new knowledge — like the semantics, orthography and pronunciation of out-of-vocabulary terms — as the system is running, particularly by a single, natural demonstration is critical to significantly enhancing the usability of observant, intelligent systems